System and Method for Compressing Video Data and Alpha Channel Data using a Single Stream

ABSTRACT

A system and method provides video data and its corresponding alpha channel data using a single stream or file, compressed using the same format and decompression algorithm for each of the video data and the alpha channel data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of, and claims priorityto, U.S. patent application Ser. No. 11/219,232, entitled System andMethod for Compressing Video Data and Alpha Channel Data Using a SingleStream, to inventor Tinic Uro, which was filed on Sep. 2, 2005. Thedisclosure of the foregoing application is incorporated herein byreference in its entirety.

FIELD OF THE INVENTION

The present invention is related to video signal processing and morespecifically to video signal encoding and decoding.

BACKGROUND OF THE INVENTION

Video content can be streamed or downloaded over the Internet using avariety of techniques. However, adding alpha channel support for videois much more involved. The alpha channel refers to the transparency ofthe video: low-value alpha channel video stream composited over anotherimage will make the video image to seem semi-transparent, allowing theuser to see the other image; at least in part, through the video.

To support alpha channel in the video can require a video codec thatcontains alpha channel support. A pair of video codecs are used ateither end, one compressing the video stream and the other decompressingthe video stream. However, the available video codecs that provide alphachannel support aren't very good at compressing the data. CODECs that donot provide alpha channel support have a much higher compression.

Some products deal with this fact by putting the uncompressed alphachannel into a separate track. Each video frame contains two tracks: oneholding the compressed video, encoded as RGB, and the other holding theuncompressed alpha channel information. This arrangement has theadvantage that the compression for the video can be exceptionally high,at least in part making up for the fact that the alpha channel track isuncompressed. However, the compression provided is less than what couldbe possible if the alpha channel was compressed. Because of the lack ofthe highest compression in CODECs that do provide alpha channel support,a designer is required to choose between two suboptimal solutions.

What is needed is a system and method that can provide alpha channelsupport for video streaming and downloading, that can provide highercompression than a conventional video codec, and that provides alphachannel support and higher compression than using a conventional codecthat does not provide alpha channel support, and therefore leaves thealpha channel data uncompressed.

SUMMARY OF INVENTION

A system and method receives video data in RGB format and alpha channeldata and encodes it into a single file or stream. The RGB format videodata is premultiplied by the alpha channel. The premultiplied, adjustedRGB data is converted into YUV-encoded video using conventionaltechniques. The YUV is then compressed using a conventional YUV codec.The alpha channel is added to the Y input of a second YUV codec (or isencoded at a different time using the same codec as is used to encodethe video data) and the resulting YUV stream is compressed and stored ina second track of the same file or stream, allowing both tracks to becompressed, but using a video codec that does not provide integratedalpha channel compression, and therefore provides higher compressionthan would be available using conventional codecs that provideintegrated alpha channel support.

The premultiplication is performed to ensure better compression: as thealpha channel is nearer to zero, the compression of the premultipliedsignal is greater than the compression of the signal that would occurwithout the premultiplication. To perform such adjustment, the systemand method takes the minimum of the computed premultiplied R and alphato be the premultiplied R, the minimum of the computed premultiplied Gand alpha to be the premultiplied G, and the minimum of the computedpremultiplied B and alpha to be the premultiplied B.

At the receiver, each track is decoded using its own YUV decompressor,or the same YUV decompressor at different times, with the Y portion ofthe second track being extracted as the alpha. The decoded premultipliedYUV is converted to RGB, clamped to not higher than the value of alpha,and divided by the alpha to regenerate at least approximately, theoriginal R, G and B. Although the UV portion of track 2 is not used, theY portion holds approximately 80% of the data, causing the compressionof the Y portion to more than compensate for the waste. Any conventionalYUV codec can be used, such as those commercially available from ON2Technologies. Formats other than YUV may also be used.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block schematic diagram of a conventional computer system.

FIG. 2 is a flowchart illustrating a method of compressing video dataand its associated alpha channel data according to one embodiment of thepresent invention.

FIG. 3, consisting of FIGS. 3A and 3B, is a flowchart illustrating amethod of decompressing video and its alpha channel data according toone embodiment of the present invention.

FIG. 4 is a block schematic diagram of a system for compressing videoand its associated alpha channel data according to one embodiment of thepresent invention.

FIG. 5 is a block schematic diagram of a system for decompressing videoand its associated alpha channel data according to one embodiment of thepresent invention.

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT

The present invention may be implemented as computer software on aconventional computer system. Referring now to FIG. 1, a conventionalcomputer system 150 for practicing the present invention is shown.Processor 160 retrieves and executes software instructions stored instorage 162 such as memory, which may be Random Access Memory (RAM) andmay control other components to perform the present invention. Storage162 may be used to store program instructions or data or both. Storage164, such as a computer disk drive or other nonvolatile storage, mayprovide storage of data or program instructions. In one embodiment,storage 164 provides longer term storage of instructions and data, withstorage 162 providing storage for data or instructions that may only berequired for a shorter time than that of storage 164. Input device 166such as a computer keyboard or mouse or both allows user input to thesystem 150. Output 168, such as a display or printer, allows the systemto provide information such as instructions, data or other informationto the user of the system 150. Storage input device 170 such as aconventional floppy disk drive or CD-ROM drive accepts via input 172computer program products 174 such as a conventional floppy disk orCD-ROM or other nonvolatile storage media that may be used to transportcomputer instructions or data to the system 150. Computer programproduct 174 has encoded thereon computer readable program code devices176, such as magnetic charges in the case of a floppy disk or opticalencodings in the case of a CD-ROM which are encoded as programinstructions, data or both to configure the computer system 150 tooperate as described below.

In one embodiment, each computer system 150 is a conventional SUNMICROSYSTEMS ULTRA 10 workstation running the SOLARIS operating systemcommercially available from SUN MICROSYSTEMS, Inc. of Mountain View,Calif., a PENTIUM-compatible personal computer system such as areavailable from DELL COMPUTER CORPORATION of Round Rock, Tex. running aversion of the WINDOWS operating system (such as 95, 98, Me, XP, NT or2000) commercially available from MICROSOFT Corporation of Redmond Wash.or a Macintosh computer system running the MACOS or OPENSTEP operatingsystem commercially available from APPLE COMPUTER CORPORATION ofCupertino, Calif. and the NETSCAPE browser commercially available fromNETSCAPE COMMUNICATIONS CORPORATION of Mountain View, Calif. or INTERNETEXPLORER browser commercially available from MICROSOFT above, althoughother systems may be used.

FIG. 2 is a flowchart illustrating a method of compressing video and itsalpha channel data according to one embodiment of the present invention.Referring now to FIG. 2, video data and associated video alpha channeldata is received 212. In one embodiment, video data includesconventional RGB formatted video data and the alpha channel dataincludes either a single alpha channel value for each frame of the videoor alpha channel values for components of the video, such as each pixelof the video data. The video data may be received 212 as a single file,or, as indicated by the dashed line in the Figure, as a continuousstream, with additional video data being received while it is beingprocessed as described herein.

The first pixel of video data received in step 212 is selected 214 andthe selected pixel of video data is premultiplied 216 by itscorresponding alpha channel data value. In one embodiment, this isaccomplished by individually multiplying the R value, G value and Bvalue of a pixel by the alpha value corresponding to that pixel. Thepremultiplied R, G and B values are referred to herein as R′, G′, andB′.

Alpha channel data values are used generally to composite the video datawith other multimedia data. The alpha channel data values for the videoare a measure of the opacity of the video. Alpha channel values of zerocause the video data to be invisible with respect to other multimediabeing displayed behind the video corresponding to the video data (e.g.an animation such as a conventional Flash animation to be displayed bythe conventional Flash player plug-in to conventional browserscommercially available from Macromedia, Inc., of San Francisco, Calif.),and alpha channel values of one cause the video data to appear to beopaque with respect to other multimedia being displayed behind the videocorresponding to that video data.

In one embodiment, each pixel is received with its own alpha channelvalue. In another embodiment, a pixel may not be received with an alphachannel value, in which case the pixel is considered to have the alphachannel value of the prior pixel until a pixel having a designated valueis received. Alternatively, the number of pixels to which an alphachannel value corresponds may be stored adjacent to that alpha channelvalue. In such embodiment, the alpha channel value of the prior pixelmay be retained and used as the alpha channel value for the pixel beingcurrently processed until a new alpha channel value is to be used forthe pixel.

The RGB values of the video are converted 230 to YUV values, eitherusing a conversion table or via calculations. Any conventionalconversion may be used, such as those described in ITU-R BT.601, whichis described at en.Wikipedia.org/wiki/YUV, anden.Wikipedia.org/wiki/CCIR_(—)601 In one embodiment, the conversion ismade as follows:

Y=0.299R+0.587G+0.114B  (Eq. 1)

U=0.492(B−Y)=−0.147R−0.289G+0.436B  (Eq. 2)

V=0.877(R−Y)=0.615R−0.515G−0.100B  (Eq. 3)

The YUV data converted in step 230 is added 232 to a compression buffer,and if there is a new alpha value for the set of pixels or an alphavalue for the pixel 234, the alpha value is added to the compressionbuffer as the Y-portion of YUV-formatted data and the method continuesat step 238. In one embodiment, if the alpha channel values are sharedamong various pixels, if the alpha channel value of a prior pixel isused, it may not be added to the compression buffer in one embodiment,such value being added only when it is initially received.

Other information may be stored in the compression buffer, and treatedas part of the video data or alpha channel data. For example, if thealpha channel value indicates a number of pixels corresponding to thealpha channel value, such information is stored with the alpha channelvalue and will be compressed along with it as described below.

At step 238, if there are no more pixels received in step 212, anindication is made that the conversion is done 238 and otherwise 238,the next pixel is selected 242 and the method continues at step 216using the selected pixel.

Compression and Building of the Frame

The compression of the video data in the compression buffer is shown inthe flowchart of FIG. 2B according to one embodiment of the presentinvention.

Referring now to FIG. 2B, data to be compressed is received 250 into abuffer. The data may be video data or alpha channel data. If a frameexists, the method continues at step 256 and otherwise, a new frame isbuilt 254. In some embodiments, there may be leftover compressed datathat was not able to be added to the prior frame. In those embodiments,step 254 includes adding such compressed data to the frame, either thevideo track or the alpha channel track, depending on whether theleftover compressed data corresponds to video or alpha channel data,respectively.

If at least some of the data received in the buffer being processed plusany remaining uncompressed data in the buffer defines a set of data thatis ready for compression 256, the method continues at step 258 andotherwise 256, the method continues at step 250, maintaining the datareceived plus any data not already compressed in the buffer for the nextiteration of step 256. In one embodiment, a set of data is ready forcompression if the data received is incompatible with an optimumcompression of the uncompressed data previously stored in the buffer. Asa simple example, in the series of data 4,4,4,4,4,8,8,8, that would becompressed as 5,4 and 3,8 to indicate 5 “4”s and 3 “8”s, as each of the‘4’s arrives, they are not ready for compression because it is unknownhow many ‘4’s there will be to allow the optimum compression to occur.However, when the first of the ‘8’s arrives, the five ‘4’s are thencompressible, and the arriving 8 will remain in the buffer.

In one embodiment, data is also ready for compression when no furtherdata will be received for the track, frame or both (e.g. the datareceived is the last piece of data, which may be indicated using aspecial value, with any actual data having that value converted to aslightly different value) or when no further data can be added to thetrack, for example, because it is full or because a logical end has beenreached.

It isn't necessary to store the video data and alpha channel data in thecompression buffer, as the compression may be performed by receivingdata for one pixel at a time and one alpha channel value at a time andsaving different state variables for each and causing the compresseddata to be written out at such time as data is received that will allowcompressed data to be written out without any loss of compression. Forexample, in the simple compression scheme in which a value is followedby the number of times that value was consecutively received in thecompressed state, the compressed data would be provided when a value isreceived that is different from the data received immediately prior.Until then, state indicating the value of the previously received data,and the number of times it was consecutively received, may be stored.Thus, the use of a compression buffer is but one way of implementing thepresent invention.

At step 258, the data received as YUV data in the buffer is output ascompressed YUV data using any conventional compression algorithm, suchas those specified by the H.263 or H.261 standards, or any conventionalcompression algorithm based upon either or both of them or derived fromeither or both of them, or using the conventional On2 Technologies VP6codec compression product commercially available from On2 Technologies,of New York, N.Y., with any uncompressed data remaining in the buffer.As noted above, two types of data are received in the buffer and thedata type received is processed only with other data of that type. Thus,video data is only processed with other video data, or state variables,(for that type of data), but not with any alpha channel data, and viceversa. If the track is done, either because it is full or a logical endof the track is indicated 260, the method continues at step 264 andotherwise 260, the compressed data is added 262 to the video or alphachannel track, depending on the buffer being processed, and the methodcontinues at step 250. Otherwise 260, any leftover compressed data isstored 264 to be added to the next frame.

If the frame is ready to be output 266, the frame is output 268, forexample, by storing it to a file or by streaming it to one or morecomputer systems and the method continues at step 250. Otherwise 266,the method continues at step 250.

In one embodiment, instead of continuing at step 250 each time, in whichcase the flow charts of FIGS. 2A and 2B can operate as independentprocesses, the method can process one pixel of video data and oneelement of alpha channel data if any, and then the method continues atstep 238. In such embodiment, in FIG. 2 A, step 250 follows step 236 orthe “no” branch of step 234, so that each pixel of video data (and anyadditional alpha channel data) is processed fully through FIGS. 2A and2B before the next pixel is processed.

FIG. 3, consisting of FIGS. 3A and 3B, is a flowchart illustrating amethod of decompressing video and its associated alpha channelinformation according to one embodiment of the present invention. Thevideo and alpha channel data decompressed using the steps of FIG. 3 maybe compressed in the manner described in FIG. 2 above. Referring now toFIG. 3, specifically, FIG. 3A, a stream or file of frames of video andalpha channel information is received 310. A frame is selected 312 andtwo pointers serving as placeholders into the data corresponding to thefirst pixel of track one and track two of the selected frame areinitialized 314.

As described above, in one embodiment, each pixel may have its own alphachannel value and in other embodiments, some alpha channel values may beshared across multiple pixels. Information may be included with eachalpha channel value or with certain pixels that designates whether a newalpha channel value is required or whether the previous one should bereused, or such information may be implied. For example, each layer mayemploy its own alpha channel value and a new one is used when the firstpixel from a new layer is selected for display. The boundary for a newlayer may be a reserved value of pixel, for example with pixels thatactually have that value being adjusted slightly.

As such, some embodiments may obtain a pixel, and then determine whetherto obtain a new alpha channel value, however other embodiments in whicha new alpha channel value is always obtained with every pixel, orembodiments in which the determination as to whether a new alpha channelvalue is to be obtained may be made without obtaining the next alphachannel value can obtain any new alpha channel value, before the nextpixel is obtained. As will now be described, the pixel is obtainedfirst, and then the alpha channel value is obtained. This exampleillustrates the desirability of identifying the proper alpha channelvalue as the pixel is being processed. If the alpha channel value isfirst obtained, processing may be easier because the alpha channel valueused to properly decode the pixel value will already be available.

In one embodiment, at any given time, there may be pixels that arecompressed, some that are decompressed but not further processed(because the decompression may decompress a batch” of pixels, only oneof which is then processed). If there is an uncompressed pixel that hasnot been otherwise processed as described herein 316, that pixel will berepresented as an uncompressed YUV-formatted pixel, and such pixel isselected 324 and converted 326 to RGB using any conventional conversionmethod, such as that specified by ITU-R BT.601, that described atwww.fourcc.org/fccyvrgb.php or equations 4 through 6 below, and themethod continues at step 330.

R=Y+1.402(U−128)  (Eq. 4)

G−Y−0.34414(V−128)−0.71414(U−128)  (Eq. 5)

B=Y+1.772(V−128)  (Eq. 6)

If there is no uncompressed video pixel 316, track 1 is selected 318 andthe compressed data for one or more pixels at the pointer for track 1 isselected, and the pointer is advanced to the data representing the nextset of one or more pixels in track 1 320. The smallest amount ofdecompressible compressed data at the pointer is decompressed 322 toproduce one or more uncompressed pixels and the method continues at step324 as described above. The decompression used may be any conventionaldecompression algorithm corresponding to the compression algorithm usedabove, such decompression algorithms including those specified by theH.263 or H.261 standards, or any conventional. algorithm based uponeither or both of them or derived from either or both of them, or may beperformed by the conventional On2 Technologies VP6 codec decompressionproduct commercially available from On2 Technologies of New York, N.Y.

Step 326 may include identifying whether a new alpha value is requiredin embodiments that employ such determination and do not supply an alphachannel value with each video pixel. In one embodiment, a pixel valuemay be reserved for the purpose of indicating that the next alphachannel value should be obtained for use with all subsequent pixelsuntil the next time the reserved value is used as a pixel. The reservedvalue is then discarded and another pixel is obtained. Pixels that havethe reserved value when received for encoding are converted to a nearbyvalue. Alternatively, each alpha channel value may indicate to how manypixels it applies.

If an alpha channel value from those unprocessed is to be used with thepixel being processed 328 if an uncompressed alpha channel value isavailable 330, the next uncompressed Y value from track 2 is selected338 and the alpha is set to that value 340 and the method continues atstep 342.

If an unprocessed alpha channel value is not needed 328, because forexample, a previous alpha channel value can be used, the methodcontinues at step 378 of FIG. 3B, described below, using the prior alphachannel value.

If a new alpha channel value is needed 328 but there are no uncompressedalpha channel values available 330, the second track is selected 332 andthe alpha channel data pointed to by the pointer at the selected trackis selected 334, the smallest decompressible set of one or more of theYUV data is decompressed 336, using any conventional decompressionalgorithm compatible with the compression, such as those specified bythe H.263 or H.261 standards, or any conventional algorithm based uponeither or both of them or derived from either or both of them, or usingthe conventional On2 Technologies VP6 codec decompression productcommercially available from On2 Technologies, New York, N.Y., and thealpha for the pixel being processed is set 340 equal to the decompressedY from the first YUV decompressed and the method continues at step 378of FIG. 3B.

In one embodiment, illustrated in FIG. 3B, a check is made to ensurethat the premultiplied R′, G′ or B′ of the pixel does not exceed thevalue of the alpha corresponding to that pixel, and if any of them do378, 382, 386, the value of the premultiplied R′, G′ or B′ that exceedsthe value of the corresponding alpha is reduced to the correspondingalpha. The method continues at step 342. If the check is not performed,step 342 follows step 340 as shown by the dashed line in the Figure orstep 342 follows the “no” branch of step 328, also shown by the dashedline in the Figure. The check “clamps” the premultiplied values of R′,G′, or B′ to not higher than alpha, to correct for any losses in thecompression that could cause any of them to have a value higher thanalpha. Reducing them to a value of alpha will cause them to be at leastcloser to their original values, after the division of step 342, below.

The uncompressed video data is the RGB data premultiplied by alpha asdescribed above, and so it is referred to herein as R′G′B′. Each of theR′G′B′ values are divided 342 by the corresponding alpha to produceunpremultiplied values that are at least approximately equal to theiroriginal counterparts received in step 212.

The alpha and video data may be formatted as described below and stored344 or provided and such storage or provision is made for the purpose ofcompositing the video data with multimedia data using the alpha channeldata as a measure of opacity of the video data with the multimedia orother information intended to be displayed beneath it. In oneembodiment, storage of the alpha channel data and video data is made byformatting it into the conventional RGBA or ARGB file format, in whicheach component, the alpha and each of the R, G and B consist of eightbits to form a 32 bit pixel, with the alpha value either at the head ortail of each pixel. RGBA is described at en.wikipedia.org/wiki/RGBA ande.wikipedia.org/wiki/RGB, with ARGB being a variant of RGBA.

If there is more unprocessed video in the frame 346, either uncompressedvideo pixels that have not been processed, or compressed video pixels,the method continues at step 316 and otherwise 346, the video may beprovided 348 via the file or by streaming it. If more frames areavailable 360, the next frame is selected 362 and the method continuesat step 314 using the newly-selected frame. Otherwise 360, the nextframe is awaited 364 and the method continues at step 360. It isn'tnecessary to provide the video when the frame has been completelyprocessed, and so in one embodiment, each pixel or a group of pixels,when, sufficient numbers of pixels have been processed as describedherein; may be provided as part of step 344.

Referring now to FIG. 4, a system for compressing, and placing into astream, video data and associated alpha channel data is shown accordingto one embodiment of the present invention. Data receiver 412 receivesvia input/output 410 video data in pixels, in RGB format, and alphachannel data. An indication as to whether the data is alpha channel dataor video data may either be expressly made as part of the data or may beimplied from the ordering of the information or by using otherconventional formatting techniques. In one embodiment, such data isreceived in the conventional ARGB or RGBA format, though otherconventional file formats, or those for storing video data, may be used.

Input/output 410 may be connected to a conventional video capture deviceor a file that contains video data in RGB format and alpha channel datafor the video data. In one embodiment, data receiver 412 includessuitable communications interface hardware and software allowing it toreceive the data in a suitable format.

As noted above, the alpha channel data may be received for each pixel ora single alpha channel value may be received that should be applied tomultiple pixels. If the data is alpha channel data, data receiver 412retains it and also adds it as the Y data of YUV data and provides theYUV formatted alpha channel data to the end of an alpha channel queue orlocation in buffer 408. Buffer 408 includes conventional memory or diskstorage, which may be arranged as two conventional queues: an alphachannel queue and a video queue, although two locations, or otherarrangements may be used. Although buffer 408 is described as a element,the buffer 408 may in fact be part of YUV compressor 420 describedbelow. In one embodiment, there are two buffers: one for the video dataand another for the alpha channel data, each one arranged as aconventional queue or two locations.

If the data is a pixel of video data, data receiver 412 provides thefirst pixel of the video data and also provides the then currentlyapplicable alpha channel data (e.g. the most recently received alphachannel data) to premultiplier 414. In embodiments in which the alphachannel data follows the video data, data receiver 412 may buffer thevideo data until it receives the alpha channel data that corresponds tothe video data, at such time providing the video data and alpha channeldata as described above.

Processing of Video Data

When it receives the video data, premultiplier 414 premultiplies each ofthe R, G and B by the alpha channel data it receives to produce R′, G′and B′ and provides the resulting R′, G′ and B′ and the alpha channeldata to RGB to YUV converter 418.

When it receives the R′G′B′, RGB to YUV converter 418 converts the R′,G′ and B′ to YUV format as described above. In one embodiment, RGB toYUV converter 418 performs the conversions from RGB to YUV describedherein. RGB to YUV converter 418 then provides the YUV data to the endof the video queue in buffer 408.

Compression of Video and Alpha Channel Data

In one embodiment, YUV compressor 420 monitors the two queues orlocations in buffer 408 and compresses the YUV video data and alphachannel data received as described above to allow for optimumcompression.

In one embodiment, two YUV compressors 420, 421 are used: one for videodata and one for alpha channel data. Each one 420, 421 obtains from theappropriate queue or location in buffer 408 the video data or alphachannel data in YUV format. When YUV data is received, the appropriateYUV compressor 420 or 421 identifies whether compressed output may bemade, and if so, outputs the compressed data to track manager 422,removes the uncompressed data corresponding to the output compresseddata from the buffer, and either retains the uncompressed data orinternally updates a state of the uncompressed data received andoptionally removes it from buffer 408. If compressed data is not readyfor output as described above, YUV compressor 420, 421 either retainsthe uncompressed data in the buffer 408, or discards it and internallyupdates the state of the uncompressed data received. In one embodiment,YUV compressor 420 performs any conventional compression algorithm, suchas those specified by the H.263 or H.261 standards, or any conventionalcompression algorithm based upon either or both of them or derived fromeither or both of them, or includes the conventional On2 TechnologiesVP6 codec compression product commercially available from On2Technologies, New York, N.Y. As noted, YUV compressor 420 compresses thealpha channel data as if it was the Y channel of YUV video data.

Storage of the Data

YUV compressor 420 provides the compressed YUV video data or thecompressed alpha channel data (and an indication as to whether the datais video data or alpha channel data) to track manager 422, which storesthe compressed YUV video into the first track of the current frame inframe storage 430 or stores the compressed YUV alpha channel data intothe second track of the current frame in frame storage 430 based on theindication it receives. In one embodiment, pointers to the currentframe, track one of that frame, and track two of that frame are storedin frame storage 430 at a known location by track manager 422 when itbuilds the frame. Track manager 422 uses the pointer for track 1 tostore in frame storage 430 the compressed YUV video and the pointer fortrack 2 to store the compressed YUV alpha channel data, and then trackmanager 42 2 updates the pointers to the track corresponding to the datait stores.

In one embodiment, when the either track of a frame is full, trackmanager 422 builds a new frame and supplies the remainder of the videodata and/or alpha channel data to the appropriate track in the framethat has capacity to accept it. As it stores data into the frame, trackmanager 422 updates the appropriate pointers for each track to allow itto identify an unused location in each track in which to storeadditional compressed data.

When a frame is full, track manager 422 completes the frame and adds thepointer to the current frame to the end of a stream/storage queue inframe storage 430, removes a pointer from a queue of pointers toavailable frames (that it initializes upon startup) in frame storage430, builds another frame into frame storage 430 if one was not alreadybuilt as. described above, and sets the pointers to the new frame, andtracks as required.

Frame output manager 424 monitors the stream/storage queue in framestorage 430 and when it identifies a pointer at the head of the queue,it either stores, via communication manager 426 the frame into a file,which may be later streamed via a network, or outputs it as a stream viainput/output 428 to a network via communication manager 426.Communication manager 426 includes a conventional communicationinterface that may include support for TCP/IP, Ethernet, and otherconventional communication protocols. The network may include a localarea network, the Internet, or both.

FIG. 5 is a block schematic diagram of a system for decompressing andproviding video data and alpha channel data according to one embodimentof the present invention. The system of FIG. 5 may, be used todecompress compressed alpha channel and video data compressed asdescribed herein. Referring now to FIG. 5, the file or stream may bereceived via input/output 506 of communication interface 508, whichincludes a conventional communication interface that may include supportfor TCP/IP, Ethernet, and other conventional communication protocols.When communication interface 508 receives the file or stream, itprovides it to file/stream receiver 510.

When it receives a file or begins to receive a stream, file/streamreceiver 510 provides the file, or a frame or frames received from thestream or file to data storage 534 with the frames arrangedsequentially. Frame selector 512 monitors data storage (which may bearranged, by file/stream receiver 510 in the portion of data storage 534reserved for the stream, as a set of frames received and pointer to oneor more frames) and selects the first frame from the stream.

Frame selector 512 initializes a set of pointers to the tracks in theframe, one corresponding to track one and one corresponding to track twoof the selected frame if track two exists (or null otherwise), placesthem in known locations in data storage 534 for the current frame andsignals YUV to RGB converter 518.

In one embodiment, track one starts every frame or is located at a fixedoffset from the start of the frame and either contains a fixed number ofbytes or an end-of-track marker. Frame selector 512 uses the location oftrack one to initialize the pointer to it and uses the length of trackone or the end-of-track marker to set the pointer to the start of tracktwo.

When signaled, YUV to RGB converter 518 identifies whether it has atleast one pixel of uncompressed, but otherwise unprocessed video data.If so, YUV to RGB converter 518 converts the next pixel of YUV-formatdata to RGB-format data, though the values are R′G′B′ values,premultiplied by alpha as described above. YUV to RGB converter 518provides the RGB-format data to clamper 522, described below anddiscards the YUV-format video data corresponding to the RGB-format datait provides.

If YUV to RGB converter 518 does not have at least one pixel ofuncompressed, but otherwise unprocessed video data, YUV to RGB converter518 signals YUV decompresor 516 with an identifier of track number oneto indicate that it requires additional video data.

When so signaled by YUV to RGB converter 518 YUV decompressor 516retrieves the smallest set of data at the track one pointer that can bedecompressed into to one or more pixels, decompresses it, and updatesthe pointer to the next set of data in the track, or provides to frameselector 512 an indicator that indicates that there is no, more data intrack one of the current frame and frame selector selects the next frameand updates the pointer to track one of that frame. YUV decompressor 516then provides the uncompressed YUV video data to YUV to RGB converter518, for use as described above.

In one embodiment YUV decompressor 516 performs any conventionaldecompression algorithm compatible with the compression, such as thosespecified by the H.263 or H.261 standards, or any conventional algorithmbased upon either or both of them or derived from either or both ofthem, or includes the conventional On2 Technologies VP6 codecdecompression product commercially available from On2 Technologies, NewYork, N.Y. and YUV to RGB converter 518 performs the conversion from YUVto RGB as described above.

Clamper 522 adjusts, if necessary, the R′, B′ and/or G′ to ensure thatnone of them exceeds the alpha channel data value as described above.Clamper 522 provides the resulting R′, G′ and B′, adjusted as required,to RGB to premultiplication remover 530.

When it receives the R′G′B′ video data as described above, in. oneembodiment, premultiplication remover 530 determines whether a new alphachannel value corresponds to the first unprocessed pixel of the videodata it received, or the same alpha channel value it used previously maybe used. In one embodiment, a predetermined R′G′B′ value is used toindicate that a new alpha channel value is used, the R′G′B′ value beinginserted when the predetermined RGB value is received, with any actualR′G′B′ that equals the predetermined value being adjusted to a nearbyvalue. If the previous alpha channel value is to be used,premultiplication remover 530 uses the premultiplied video data itreceives and the prior alpha channel information as described above anddivides each of the premultiplied R′, G′, and B′ by the correspondingalpha channel data for that pixel, to produce an R, G and B that atleast approximately corresponds to the R, G and B that had been receivedat data receiver 412 of FIG. 4. In one embodiment, premultiplicationremover 530 stores the alpha for the next set of video data it receives,for reuse. Premultiplication remover 530 signals YUV to RGB converter518 and provides the resulting pixel to data storage manager 532.

If a different alpha channel value is required, premultiplicationremover 530 determines whether it has received the alpha channel valuefrom YUV decompressor 516. If so, it selects the next alpha channelvalue in the order in which it was received and provides it to alphaextractor 520. If not, it signals YUV decompressor 516 with anidentifier of track two, to indicate that the next set of one or moredecompressed alpha channel value at the track two pointer is required.YUV decompressor 516 selects the next set of data that will decompressinto the minimum number of YUV data at the track two pointer,decompresses the data, provides the decompressed data topremultiplication remover 530, and updates the track two pointer.Premultiplication remover 530 then stores the YUV data, selects thefirst YUV-formatted pixel and provides it to alpha extractor 520.

Alpha extractor 520 receives the uncompressed YUV containing the alphachannel data, and extracts the Y channel from the YUV formattedinformation it receives as the alpha channel data. Alpha extractor 520then provides the alpha channel data to premultiplication remover 530for use as described above and to data storage manager 532 for use asdescribed below. When premultiplication remover 530 receives the alphachannel data, it processes it as described above, signals YUV to RGBconverter 518, and provides the resulting pixel to data storage manager532 when signaled, YUV to RBG converter 518 repeats the process.

Although the use of a single YUV decompressor is described herein todecompress both the video data and the alpha channel data, in anotherembodiment, two or more decompressors are used that can simultaneouslydecompress the YUV data from the two tracks. However, in one embodiment,all decompressors used to decompress video data and alpha channel dataemploy the same decompression algorithm and the video and alpha channeldata are, for a time, in the same format.

Data storage manager 532 receives the video data and alpha channel datafor each of the pixels and formats them into a data structure or a filein data storage 534. In one embodiment, data storage manager 532 formatsthe video data and alpha channel data into the conventional RGBA or ARGBfile format described above. The file can then be used by a conventionalmultimedia program such as the Flash Player plug-in to conventionalbrowsers, to integrate the video with other multimedia, such asconventional Flash animations. The alpha channel is used to compositethe video data with one or more layers of multimedia images underneaththe video data as described above.

1. A method of providing video data and alpha channel data, comprising:receiving the video data in a first format; receiving the alpha channeldata in the first format; separately compressing the video data andalpha channel data using a single compression algorithm to compress eachof the video data and the alpha channel data; and providing thecompressed video data and compressed alpha channel data in one selectedfrom a single stream and a single file.
 2. The method of claim 1 whereinthe format comprises YUV.
 3. The method of claim 2, wherein the alphachannel data is placed in the Y portion of the YUV format.
 4. The methodof claim 1, wherein the video data received comprises pixels, comprisingRGB data, and the method additionally comprises: multiplying each of theR, G and B of the pixels of the video data by the alpha channel datacorresponding to each said pixel; and converting the multiplied R, G andB into the first format for each said pixel.
 5. A system for providingvideo data and alpha channel data, comprising: storage; and a processorcoupled with the storage and configured to effect: a data receiverhaving a first input coupled for receiving the video data in a firstformat and a second input coupled for receiving the alpha channel datain the first format, the data receiver for providing the video data andalpha channel data at an output, and at least one compressor having aninput coupled to the data receiver output for receiving the video dataand alpha channel data, the at least one compressor for separatelycompressing the video data and alpha channel data using a singlecompression algorithm for each of the video data and the alpha channeldata, the compressor for providing, via an output, the compressed videodata and compressed alpha channel data in one selected from a singlestream and a single file.
 6. The system of claim 5 wherein the formatcomprises YUV.
 7. The system of claim 6, wherein the alpha channel datais placed in the Y portion of the YUV format.
 8. The system of claim 5,wherein the video data received comprises pixels, comprising RGB data,and the system additionally comprises: a premultiplier coupled betweenthe data receiver output, via a premultiplier input, and the compressorinput, via a premultiplier output, the premultiplier for multiplyingeach of the R, G and B of the pixels of the video data by the alphachannel data corresponding to each said pixel; and a converter coupledbetween the premultiplier output, via a converter input, and thecompressor input, via a converter output, the converter for convertingthe multiplied R, G and B into the first format for each said pixel. 9.The system of claim 8 additionally comprising an adjustor coupledbetween the premultiplier output, via an adjustor input, and thecompressor input, via a premultiplier output, the adjustor for adjustingat least one of the multiplied R, G or B of at least one pixelresponsive to: the alpha channel data corresponding to the pixel; andsaid multiplied R, G or B.
 10. A computer program product comprising acomputer useable medium having computer readable program code embodiedtherein for providing video data and alpha channel data, the computerprogram product comprising computer readable program code devicesconfigured to cause a computer system to: receive the video data in afirst format; receive the alpha channel data in the first format;separately compress the video data and alpha channel data using a singlecompression algorithm to compress each of the video data and the alphachannel data; and provide the compressed video data and compressed alphachannel data in one selected from a single stream and a single file. 11.The computer program product of claim 10 wherein the format comprisesYUV.
 12. The computer program product of claim 11, wherein the alphachannel data is placed in the Y portion of the YUV format.
 13. Thecomputer program product of claim 12, wherein the video data receivedcomprises pixels, comprising RGB data, and the computer program productadditionally comprises computer readable program code devices configuredto cause the computer system to: multiply each of the R, G and B of thepixels of the video data by the alpha channel data corresponding to eachsaid pixel; and convert the multiplied R, G and B into the first formatfor each said pixel.
 14. The computer program product of claim 13additionally comprising the computer program product comprising computerreadable program code devices configured to cause the computer system toadjust at least one of the multiplied R, G or B of at least one pixelresponsive to: the alpha channel data corresponding to the pixel; andsaid multiplied R, G or B.